Display Management Server

ABSTRACT

A display management unit configured to provide a modified video signal for display on a target display over an electronic distribution network. The unit may access information regarding the target display and at least one input. The unit comprises a database interface configured to retrieve display characteristics corresponding to the information regarding the target display from a characteristics database, and a mapping unit configured to map at least one of tone and color values from the at least one input to corresponding mapped values based at least in part on the retrieved display characteristics to produce the modified video signal.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.13/810,734, filed Jan. 17, 2013, which is the United States nationalstage of International Patent Application No. PCT/US2011/044616, filedJul. 20, 2011, which claims priority to United States Provisional PatentApplication No. 61/366,899, filed Jul. 22, 2010, all of which are herebyincorporated by reference in their entirety.

TECHNICAL FIELD

The invention relates to the creation, processing, distribution and/orviewing of digital images, including video images. Embodiments providefunctional units that perform one or more of tone mapping and gamutmapping. Some embodiments provide server-based functional units.

BACKGROUND

The creator of a video production typically controls tones and colors sothat, when viewed, the video content has a desired appearance as guidedby the creator's creative intent. For example, a creator may wish somescenes to have a darker, more oppressive, feel than others. This mayinclude performing color grading (or ‘color timing’) on the source videodata.

Various display technologies are now available. For example, there areCRT-based displays, plasma displays, displays backlit by a variety oftypes of light sources including LEDs of various types, fluorescentlamps, high-intensity incandescent lamps, digital cinema displays etc. Aparticular display combines display hardware with video signalprocessing components that receive video signals and drive the displayhardware to display video content of the video signals.

Different displays may vary significantly with respect to features suchas:

-   -   the color gamut that can be reproduced by the display;    -   the maximum brightness achievable;    -   contrast ratio;    -   resolution;    -   acceptable input signal formats;    -   color depth;    -   white level;    -   black level;    -   white point;    -   grey steps;    -   etc.        Consequently, the same video content may appear different when        played back on different displays. Video content displayed on        some displays without modification may depart from the creator's        creative intent in one or more ways.

Some current displays can outperform displays that were state-of-the-artat a time when certain video content was created. For example, thedisplays may be capable of providing images that have brighterhighlights, greater contrast, and/or wider color gamuts than legacydisplays that were originally available when the video production wascreated. It would be desirable to take advantage of these improvedcapabilities without departing significantly from the creative intentembodied in the video content.

It may be desirable to play video content created to take advantage ofhigh-performance displays on legacy displays or displays that have lowercapabilities. It would be desirable to provide methods and apparatus foradapting the way that the video is played to preserve as much aspossible a creative intent embodied in the video content.

Different viewing environments may also cause video content to beperceived differently by viewers. For example, viewers may perceive thesame video presentation presented on the same display differentlydepending on whether the display is being viewed in a dark room or in abrightly lit room. It would be desirable to adapt the playback of videocontent taking into account the environment in which the video contentis being viewed to preserve as much as possible a creative intentembodied in the video content.

There remains a need for apparatus and methods which may be applied toadjust video signals such that the video content has a desiredappearance when played.

SUMMARY

This invention has a number of aspects. One aspect provides functionalunits that take a video signal as input, and perform color mappingand/or tone mapping on the video signal to produce an output signal. Thefunctional units may be provided upstream of an electronic distributionnetwork through which video content is provided to displays or which thevideo content is to be viewed. The functional units may be under thecontrol of the provider of video content. The functional units mayreceive various inputs as described herein and may be configured to varytheir operation based upon the inputs. The functional units may beconnected to a database to retrieve information used to vary theiroperation. Such functional units may be embedded in networked devicesthat handle video content such as video recorders, video editors, videoprocessors, video servers, video content creation tools, mobile devices,video editing stations, and the like.

Such functional units may be implemented by hardware devices,programmable data processors executing software (or firmware),configurable hardware (such as FPGAs) and suitable combinations thereof.Software that, when executed, causes a data processor to implement afunctional unit as described herein may be of a wide variety of typesincluding operating system software, image rendering software, videoplayer software, image creation software tools, video processingsoftware, and others.

Another aspect provides tone and/or color mapping methods, units andapparatus that incorporate such units wherein mappings performed by thetone and/or color mapping units are determined at least in part bycontrol signals that may be embedded in a video signal or otherwiseassociated with the video signal or by information retrieved from adatabase based on signals received from displays. In some embodiments,color and/or tone mappings are selected according to metadata embeddedin, packaged with, or delivered in parallel with a video signal. In someembodiments, color and/or tone mappings are retrieved from a databaseindexed by display class. Some embodiments provide servers or otherapparatus that may be connected to process video data en route to adisplay which embody such tone and/or color mapping units and methods.In some embodiments the control signals include data indicatingreference ambient conditions and the color and/or tone mappings areselected and/or set based at least in part on differences between thereference ambient conditions and ambient conditions as measured by oneor more ambient sensors. The ambient sensors may, for example, detectbrightness and/or color of ambient light at the display.

Another aspect provides mapping for features of video signals (thefeatures may include, for example, one or more tone and colorcoordinates). The mapping may be implemented by methods and apparatuswhich vary the mappings over time. The mappings may be varied to takeinto account one or more of: ambient conditions and the expectedadaptation of the observers' human visual systems to light. In someembodiments, the rate of change of mappings is limited so as to avoidsudden changes that would be noticeable to observers. For exampleparameters which define a mapping may be varied smoothly over time.

In some embodiments, color mapping is based at least in part on a colorof ambient light detected at a display or assumed to be present at thedisplay. In some embodiments, color mapping is varied based on thebrightness of ambient light at a display. In some embodiments, mappingsof one or more features are controlled based on sensor measurements ofambient conditions (which may, for example, include one or more ofbrightness of ambient light, color of ambient light, brightness ofambient light in a number of different color bands etc.) as well as onmetadata that specifies how the mapping ought to be set in response tothe sensor measurements.

Another aspect provides a video processing apparatus that is connectableupstream from a video display and configured to receive an input videosignal, and to compress and/or expand a gamut and/or tone of the inputvideo signal to match capabilities of the display to yield an outputvideo signal and to pass the output video signal downstream to thedisplay. The video processing apparatus may include a communicationinterface for communicating with the display and receiving from thedisplay data indicative of the capabilities of the display. A controlsystem may set tone and gamut mapping to match the capabilities of thedisplay. The control system may retrieve information for setting toneand gamut mapping from a database.

Another aspect provides mapping for features of video signalsimplemented by methods and apparatus which are configured to selectand/or determine the mappings based at least in part on capabilities ofa target display. The apparatus may determine capabilities of the targetdisplay, for example, based on user input, data relating to the targetdisplay (the data may be retrieved from or obtained from the targetdisplay itself, for example), inferred from behavior of the targetdisplay, or the like. In some embodiments selecting and/or determiningmappings comprises an interpolation/extrapolation between two videosignals representing the same video images. Theinterpolation/extrapolation may be based on capabilities of the targetdisplay relative to capabilities associated with the two video signals.

Further aspects of the invention and features of specific embodiments ofthe invention are described below and/or illustrated in the accompanyingdrawings.

BRIEF DESCRIPTION OF DRAWINGS

The accompanying drawings illustrate non-limiting example embodiments ofthe invention.

FIG. 1 is a block diagram illustrating some possible inputs for adisplay management unit.

FIGS. 2A to 2F are graphs showing example gamut mapping and/or tonemapping curves for generating modified video data.

FIG. 3 is a block diagram illustrating an example display managementunit.

FIG. 3A is a block diagram illustrating another example displaymanagement unit.

FIG. 4A is a block diagram illustrating the components of an apparatusfor implementing mapping based on two or more input video signals.

FIG. 4B is a flowchart illustrating an example method for mapping basedon two or more input video signals.

FIG. 4C is a graph illustrating a mode of interpolation/extrapolationfrom two input video signals.

FIG. 5 is a block diagram illustrating functional blocks of an exampledisplay management unit.

FIG. 6 is a block diagram illustrating a video pipeline in which displaymanagement units are applied upstream from a distribution network.

FIG. 7 is a block diagram illustrating an example transcoder whichincludes a display management unit.

FIG. 8 is a block diagram of a server-based display management system incommunication with a local display control.

FIG. 9 is a flowchart illustrating an example method for providing videocontent from a display management server to a display.

DESCRIPTION

Throughout the following description, specific details are set forth inorder to provide a more thorough understanding of the invention.However, the invention may be practiced without these particulars. Inother instances, well known elements have not been shown or described indetail to avoid unnecessarily obscuring the invention. Accordingly, thespecification and drawings are to be regarded in an illustrative, ratherthan a restrictive, sense.

FIG. 1 is a block diagram illustrating an example display managementunit 10. Display management unit 10 has an input 12 for receiving avideo signal 13 (typically in a digital format) and an output 14 forproviding a modified video signal 15 to a display (not shown). Displaymanagement unit 10 may be located upstream of an electronic distributionnetwork (such as, for example, the Internet, an Intranet, a wide areanetwork (WAN), a local area network (LAN), or the like) which is used todistribute modified video signal 15 to the display. For example, in someembodiments, display management unit 10 may be implemented in a server.

Display management unit 10 generates modified video signal 15 by mappingtone and/or color of input video signal 13 according to a tone map 17and a color map 18 respectively. In some embodiments, tone map 17 andcolor map 18 are combined and mapping of both color and tone may beperformed using a single combined map. Mapping of color and tone may beperformed simultaneously or in sequence.

In the FIG. 1 embodiment, display management unit 10 has one or moreadditional inputs 22A-22E (collectively, inputs 22) capable of receivingsignals that may be used to control the mapping of tone and/or color indisplay management unit 10. Some embodiments may not include some ofinputs 22.

The illustrated embodiment provides:

-   -   a second video signal input 22A for receiving a second video        signal;    -   one or more ambient inputs 22B for receiving information from        the display characterizing ambient conditions (e.g. one or more        of brightness of ambient light, color of ambient light,        direction of ambient light relative to the display) at a display        where the video content will be viewed;    -   a metadata input 22C for receiving metadata characterizing one        or more aspects of a creative intent that affect how the video        content ought to be displayed (e.g., the subject-matter of the        content embodied in a video signal, such as, for example,        “sports”, “movie”, or the like);    -   a control input 22D for receiving control information for        display management unit 10;    -   a display input 22E for receiving information from the display        regarding the capabilities and/or current status of the display        on which the video content is to be displayed.

Display management unit 10 controls the characteristics of tone map 17and/or color map 18 based on signals received at one or more of inputs12 and 22 and passes the resulting modified video signal 15 to output14. Display management unit 10 also has a second optional video signaloutput 25 for carrying enhanced video information 27. Enhanced videoinformation 27 may comprise, for example, information such as, highspatial frequency information, enhanced dynamic range information or thelike, and a metadata output 26 for carrying metadata 28 relating to thevideo content being managed.

Inputs 12, 22A, 22C, and 22D may, for example, be provided by sourcevideo prepared by a colorist or other post-production video professionalwhich aims to preserve the creative intent of the creator of the videocontent. Inputs 22B and 22E may, for example, be provided over theInternet from the display on which the content is to be displayed.Similarly, outputs 14, 25 and 26 may respectively send modified videosignal 15, enhanced video signal 27 and metadata 28 to the display overthe Internet. In some embodiments, ambient inputs 22B may be omitted,and adjustment of video data to account for ambient viewing conditionsmay be performed locally at the display on which the video content is tobe displayed.

In some embodiments, display management unit 10 retrieves displaycharacteristics from a display characteristics database 11 based oninformation received at display input 22E (such as, for example, adisplay class and/or individual display parameters) to control thecharacteristics of tone map 17 and/or color map 18. Displaycharacteristics database 11 may, for example, store characteristics foreach of a plurality of different classes of displays such as one or morecolor and tone maps for each class. For example, database 11 may storeone or more color or tone maps for a “Rec. 709” class of displaysconfigured to conform to ITU-R Recommendation BT.709, one or more“mobile” classes of displays configured to conform to one or morestandardized mobile display formats, classes of displays configured toconform to other standardized display formats, and one or more“advanced” class of displays which have increased capabilities ascompared to certain standardized formats. Advanced class displays mayconform to standards such as DCI P3 (SMTE-431-2) and/or Adobe RGB, forexample. Display characteristics database 11 may also store informationfor adjusting color and tone maps for a class of display based onindividual display parameters for a particular display which describehow that display may vary from a typical display of the same class.Display characteristics database 11 may, for example, be indexed bydisplay class and/or by individual display identifiers.

Display characteristics database 11 may be pre-populated with variousindividual display characteristics. Display characteristics database 11may be populated with data from display manufacturers, distributionnetworks, third parties display data suppliers, users and/or the like.Display characteristics database 11 may be updated by display managementunit 10 as new and/or updated display characteristics for individualdisplays become available.

The inputs and outputs of display management unit 10 are logicallyseparate but are not necessarily physically separate. Two or moredistinct input signals may be received at a single physical input orpresented at a single physical output. For example, metadata may beembedded in a video signal such that a single physical input may serveas both input 12 and metadata input 20C. In such cases displaymanagement unit 10 comprises a suitable mechanism for separatingdifferent input signals that may be received at the same physical input.

One circumstance in which display management unit 10 may be used is thecase where video content is to be displayed on a display capable ofreproducing colors in a gamut that may be different from that of thevideo data presented at input 12 and/or having a dynamic range that maybe different from that of the video data presented at input 12.

As one example, video data may be received at input 12 in a format thatspecifies color in a gamut broader than that of a target device such asa display and/or a dynamic range greater than that which the targetdevice is capable of reproducing. FIGS. 2A, 2B and 2C are curves thatillustrate three alternative ways to handle the mapping of parameters ofa video signal. Such parameters may comprise, for example, parameterssuch as luminance, contrast, saturation, hue, tone, etc. The approachesillustrated in FIGS. 2A to 2C may be applied, for example to gamutmapping and/or tone mapping.

Curve 30A of FIG. 2A relates a source range of video parameters (suchas, for example, luminance or gamut) in a video signal received at input12 to a target range for a video signal presented at output 14. The axesin the graph of FIG. 2A, as well as in the graphs of FIGS. 2B-F, may be,for example linear or logarithmic. In the FIG. 2A example, a first range31 of source values in the input signal is mapped to a second range 32of target values of the output signal. The values may, for example,comprise values of a color attribute or luminance values. Any values inthe input signal beyond first range 31 are clamped to the same value 33in the output signal. Value 33 may, for example, be a maximum luminanceof the display on which the video content is to be displayed. Firstrange 31 and second range 32 may be equal. In some embodiments, sourcevalues in first range 31 are unaltered by the mapping.

For example, in a case where a target display is capable of reproducingluminances up to some maximum value, for example 600 cd/m², and theinput signal contains luminance values exceeding that maximum value thenvalues up to 600 cd/m² may be unaltered in mapping to the output signalwhile any values exceeding 600 cd/m² may be set to 600 cd/m² in mappingto the output signal. A mapping like that represented by curve 30A maybe called a ‘hard clipping’ mapping. Hard clipping mappings may beapplied to tone values, color values or both (as well as otherparameters).

When hard clipping color values, the value of one channel (e.g., one ofR, G or B) is out of range, the values of all channels may be scaled bythe same ratio so that all channels are within range. In a linear RGBspace, this approximately maintains the hue. In other color spacesdifferent techniques may be used to preserve the color. For example, inan IPT space (in which color points are specified by hue, saturation andlightness values) the color may be hard clipped, for example, byclipping only the saturation value, while leaving the hue and lightnessconstant.

Curve 30B shown in FIG. 2B illustrates a soft compression embodiment. Inthis case, a first range 34 of values in the input signal is mapped to asecond range 35 of values of the output signal. Any values in the inputsignal beyond first range 34 are compressed to values that are less thanor equal to a maximum value 36 of the output signal.

Curve 30C shown in FIG. 2C illustrates a soft clipping embodiment. Inthis case, a first range 40 of values in the input signal is mapped to asecond range 41 of values of the output signal. Ranges 40 and 41 may bethe same. In some embodiments input values in range 41 are unaltered bythe mapping. For a second range 42 of values in the input signal theoutput signal is compressed to values that less than or equal to themaximum value 43 of the output signal. Any values in the input signalbeyond second range 42 are clamped to a maximum value 43 in the outputsignal.

The above examples show clipping and compression applied at the upperend of the input range. In other embodiments, clipping and/orcompression may be applied at the lower end of the input range. Also, insome embodiments clipping and/or compression may be applied at both theupper and lower ends of the input range.

Mappings like those illustrated in FIGS. 2A through 2C may beimplemented, for example, by providing a look up table. Color orluminance values for an output video signal may be obtained by usingcorresponding values for an input video signal as keys to search thelook up table. A display management unit 10 may comprise one or morelookup tables for this purpose. In some embodiments, a plurality oflookup tables are provided for a plurality of image characteristics. Forexample, a display management unit 10 may comprise a luminance lookuptable for looking up luminance values for an output video signal and oneor more color lookup tables for looking up color values for the outputsignal. In other embodiments, mapping may be implemented by providingprocessing elements and/or logic elements configured to calculate outputvalues from input values using a suitable mapping algorithm that takesinput values as inputs and produces output values as outputs. Such amapping algorithm may involve computing values for a function thatincludes a mathematical specification of a mapping curve. Some othernon-limiting example mapping algorithms are discussed below.

Static mappings may also be used in cases where video data is receivedat input 12 in a format that specifies color in a gamut narrower thanthat of a display and/or a dynamic range smaller than that which thedisplay is capable of reproducing. In such case, the lookup table(s) mayprovide output values having a larger range than the corresponding inputvalues. In some cases the output values may be outside of a range ofvalues valid for the original video data. FIGS. 2D, 2E and 2F showexample curves 30D, 30E and 30F. As one skilled in the art willappreciate, the curves shown FIGS. 2A-2F are for exemplary purposesonly, and any of a variety of known mapping curves could also beapplied. Actual mapping curves may be configured to avoid sharp cornersin some embodiments.

Each of mapping curves 30A through 30F may be characterized by one ormore parameters. For example, curve 30A may be characterized by aparameter P1 which both indicates the value to which the output signalis clipped and indicates the boundary between first and second ranges 31and 32. Curve 30B may be characterized by parameters P2 and P3. P2indicates the input value for which the output begins to be compressedand P3 indicates the maximum output value for the output signal.Additional parameters may control the form of the compression. Ingeneral, a mapping curve of almost any shape may be characterized by asufficient number of parameters.

Apparatus according to embodiments of the invention may map according toany suitable mapping curves. In some embodiments it may be desirable toimplement mapping curves of types that have been studied in theliterature. One example of such curves is Reinhard's tone mapping curve.

Mappings between values in an input signal and corresponding values inan output signal may be performed by providing logic circuits and/orprogrammed processors configured to apply suitable algorithms todetermine output values corresponding to input values. The operation ofsuch algorithms may also be controlled by setting parameters.

In general, the translation of values from the input signal tocorresponding values in the output signal may be represented by afunction:

V _(OUT) =F(V _(IN) ,P1,P2, . . . , PN)   (1)

Where V_(OUT) is an output value, V_(IN) is the corresponding inputvalue, and P1 to PN are parameters of the function F. Differentfunctions may have different numbers of parameters.

Some embodiments provide fixed display management units which adjustincoming video data in a manner that is independent of the image contentof the video data. Some embodiments are configurable to operate in oneof a number of different modes. For example, some embodiments may beconfigured to perform mappings like those illustrated in a selected oneof two or more of FIGS. 2A to 2F, or other mappings. In suchembodiments, the mapping may be changed in response to factors such asuser inputs, changes in ambient conditions, metadata or other controlinformation or the like.

FIG. 3 shows a display management unit 10A having an examplearchitecture. Display management unit 10A comprises a translation block50 and a control block 52. Translation block 50 operates to generate anoutput signal 15 at output 14. The manner of performing the conversion(e.g. the choice of F) and the parameters (e.g. values for P1 to PN) aredetermined by control block 52. Control block 52 may provide controlover the operation of translation block 50, for example, by settingvalues for parameters 54, preparing lookup tables 55 for use bytranslation block 50, or the like.

Control block 52 attempts to optimize the translation provided bytranslation block 50 for the destination of modified video signal 15. Toachieve this, control block 52 may compare explicitly communicatedand/or assumed characteristics of the video content of the input videosignal to explicitly communicated and/or assumed characteristics of thedestination of modified video signal 15. In some embodiments,characteristics of the video content are explicitly communicated bymetadata, either transmitted alongside the video signal or included withthe video signal. In some embodiments, characteristics of thedestination (such as display capabilities and viewing environment) arecommunicated by display parameters provided to data management unit 10A.In some embodiments, assumed characteristics may be preprogrammed into amemory of display management unit 10A, may be obtained throughcalibration of display management unit 10A, or may be specified by auser input. In some embodiments, the characteristics are luminance rangesupported. In some embodiments the characteristics are color gamutsupported.

In some embodiments, control block 52 may be in communication with adisplay characteristics database 51 which stores characteristics suchas, for example, information relating to parameters 54 and/or lookuptables 55 for a plurality of different classes of display. Database 51may also store information for adjusting or generating parameters 54and/or lookup tables 55 for a class of display based on individualdisplay parameters such as, for example, capabilities or viewingenvironments. In such embodiments, control block 52 receives a displayclass 53A, and optionally individual parameters 53B from the display (orfrom another device such as a set-top box or the like coupled to thedisplay), and queries database 51 to retrieve corresponding informationfor creating or modifying parameters 54 and/or lookup tables 55. Forexample, control block 52 may receive information from a displayindicating that the display belongs to a “Rec. 709” class of displays,and retrieve from database 51 corresponding parameters 54 and/or lookuptables 55 such that transformation block 50 generates an output signalsuitable for a Rec. 709 display. Control block 52 may also receiveinformation from the display indicating local deviations from standardRec. 709 display characteristics, and retrieve corresponding informationfrom database 51 for suitably modifying the operation of transformationblock 50. Local deviations from standard display characteristics mayinclude, for example, ambient level, black level, white level, whitepoint, color gamut primaries, chromaticities of color gamut primaries,and the like.

For example, control block 52 may determine that the destination devicecannot fully reproduce one or more values from the input signal and, inresponse, select or create mappings for the values that are likemappings shown in one of FIGS. 2A, 2B or 2C. As another example, controlblock 52 may determine that the destination device has the ability toreproduce a wider gamut and/or a greater dynamic range than the devicethat the input signal was created for. In response, control block 52 mayselect or create mappings for the values that are like mappings shown inone of FIGS. 2D, 2E or 2F. Control block 52 may also set parametervalues for the chosen mapping based upon the explicitly communicated,assumed and/or retrieved characteristics of the input video content andthe destination device.

In some embodiments lookup table values or other parameters that definemappings are provided in metadata. In some embodiments, lookup tablevalues or other parameters that define mappings are stored as entries ina database. A display management unit may be configured to retrieve themetadata or database entries and to set up a mapping based on themetadata or database entries. In some embodiments the metadata ordatabase entries may provide alternative mappings for a range ofapplications and the display management unit may be configured to selectcertain mappings from the metadata or database entries and apply thosemappings. For example, metadata or database entries may includedefinitions of mappings (which may consist of or comprise parametervalues for mappings in some embodiments and may consist of or comprisefuller specifications of mapping curves in other embodiments) for:

-   -   different ambient conditions that may be present at a        destination device, such as different brightness levels,        different colors of ambient light, or the like;    -   different target devices;    -   different types of media content (e.g. types of media content        having different ranges of luminance and color);    -   different portions of a video (e.g. different frames, different        scenes or the like);    -   and the like.

Control block 52 may control the initialization of translation block 50to provide appropriate mapping(s) and, in some embodiments, may modifythe mapping(s) over time or retrieve different mappings in response tosignals received at display management unit 10A. In some embodiments,control block 52 changes mappings in real time in response to changingcharacteristics of the destination device (such as, for example, ambientviewing conditions) or changing characteristics of video content.

FIG. 3A illustrates an alternative display management unit 10′ whichoperates on a single input video signal in a fixed manner The videosignal is received at input 53 and attribute values (for example, RGBvalues or LUV values or XYZ tristimulus values or IPT values etc.) areextracted by a decoder 54. Parallel translation mechanisms 55A, 55B and55C (collectively, translation mechanisms 55) map values for acorresponding attribute to mapped values. Translation mechanisms 55 may,for example, comprise lookup tables, calculation logic and/or programmedprocessors that determine mapped values as a function of values from theincoming video data or the like. An output block 56 writes the mappedvalues into output video data. Output block 56 may also optionallyencode the output video data for distribution over the Internet.

Returning to FIG. 1, display management unit 10 may provide anadditional mode of operation for cases in which different video signalseach carrying a different version of the same video content are receivedat inputs 12 and 22A. The two different versions may, for example, beindividually optimized for displays having different color gamuts and/ordynamic range. If either version of the video signal has been optimizedin a manner suitable for the destination device then that version may bepassed to the destination device without alteration. In other cases,controller 52 may derive output signal 15 by interpolating between thetwo input video signals or extrapolating from the two input videosignals. In cases where the two input video signals have each beenoptimized to preserve a creative intent (for example by suitable colortiming), this mode has the advantage that the interpolated orextrapolated values may be expected to preserve that creative intentmore closely than mappings based on one or the other version of inputvideo.

FIG. 4A is a block diagram illustrating the components of apparatus 60for implementing mapping based on two or more input video signals. FIG.4B is a flowchart illustrating a method 70 for mapping based on two ormore input video signals. Apparatus 60 comprises first and second inputs61A and 61B which carry first and second video signals, respectively. Avalue extractor 62 is configured to extract corresponding values fromthe video signals (step 72 in FIG. 4B) and to pass the values to aninterpolation/extrapolation block 64. In some embodiments the valueextractor extracts pixel values from the video signals. The pixel valuesmay, for example, comprise values of a tone and/or color coordinates ina color space. In some embodiments the color space is a perceptuallylinear color space.

Interpolation/extrapolation block 64 is configured to generate an outputvalue 65 by interpolating or extrapolating (step 73) the extractedvalues and to provide the output value in a modified video signal (step74) at an output 66. A control input 67 of interpolation/extrapolationblock 64 receives a control value that controls the operation ofinterpolation/extrapolation block 64. Control values may becharacteristic of the capabilities of and/or ambient conditions at atarget device for which output video 65 is generated, for example.Control values may be derived from or embodied in information from thedisplay, a display characteristics database, or the like. In a simpleembodiment the control value is applied in interpolation/extrapolationblock 64 to multiply a difference between the corresponding values inthe first and second video signals in a calculation of a linearinterpolation/extrapolation. The interpolation/extrapolation performedby block 64 may be based upon a value or values received at controlinput 67.

FIG. 4C illustrates one way in which the interpolation/extrapolation maybe performed. In FIG. 4C, a value in the first video signal is y₁ andthe corresponding value in the second video signal is y₂. The horizontalaxis in FIG. 4C indicates a target device capability. For example, thehorizontal axis may indicate a dynamic range of a target device, or mayindicate a logarithm of a luminance of a target device, according tosome index, with the first video signal optimized for use with a targetdevice having a capability A and the second video signal optimized foruse with a target device having a capability B. If the target device forthe video signal has the capability C then an output value may bedetermined, for example, according to:

$\begin{matrix}{V_{OUT} = {y_{1} + {\left( \frac{y_{2} - y_{1}}{B - A} \right)\left( {C - A} \right)}}} & (2)\end{matrix}$

(C−A) may be positive or negative. |C−A| may be greater or less than|B−A|.

A feature of Equation (2) is that V_(OUT) is the same as y₁ and y₂ ify₁=y₂. Thus, if a creator has deliberately set y₁ and y₂ to be the samein the first and second video signals then embodiments which applyEquation (2), as well as other embodiments that share this feature, willpreserve this value in the output video signal. On the other hand, wherey₁ y₂ V_(OUT) may be different from both of y₁ and y₂.

In some embodiments, the display management unit is configured tointerpolate/extrapolate following perceptual curves. For example, thedisplay management unit may be configured to interpolate/extrapolate onthe logarithm of the luminance. In some embodiments, video data isconverted into a perceptually uniform color space (such as, for example,IPT or LogYuv) before performing interpolation/extrapolation.

In some embodiments, interpolation between video data may be performedaccording to the following equations:

Y _(MDR) =e ^((α log(Y) ^(LDR) ^()+β log(Y) ^(VDR) ⁾⁾   (3)

x _(MDR) =αx _(LDR) +βx _(VDR)   (4)

y _(MDR) =αy _(LDR) +βy _(VDR)   (5)

where: Y is a luminance coordinate in a color space such as LogYxy orLogY u′v′; the subscript VDR (as in Y_(VDR), for example) identifiesdata from the video signal having the higher capability (e.g. higherdynamic range and/or broader color gamut); the subscript LDR identifiesdata from the video signal having the lower capability; the subscriptMDR identifies the output or target video signal; α and β areinterpolation constants; and x and y are color coordinates (for example,x and y in a LogYxy color space or u′ and v′ in a LogYu′v′ color space).Here, α may be given, for example, by:

$\begin{matrix}{\alpha = \frac{{\log \left( {\max \left( Y_{VDR} \right)} \right)} - {\log \left( {\max \left( Y_{TARGET} \right)} \right)}}{{\log \left( {\max \left( Y_{VDR} \right)} \right)} - {\log \left( {\max \left( Y_{LDR} \right)} \right)}}} & (6)\end{matrix}$

where: max (Y_(VDR)) is the maximum value for Y that can be specified inthe VDR (higher-capability) video signal, max Y_(LDR) is the maximumvalue for Y that can be specified in the LDR (lower-capability) videosignal, and max (Y_(TARGET)) is the maximum value for Y that can bereproduced on the target display. β may be given by β=1−α.

In some embodiments, the video signal having the lower capabilitycomprises a Rec. 709 video signal. In some embodiments the video signalhaving the higher capability comprises color ranges sufficient tospecify colors in a gamut that encompasses all colors perceptible by thehuman visual system.

In some embodiments, the display management unit is configured todetermine whether the capability of a target device matches (eitherexactly or to within some threshold) the capability for which the firstor second video signal is optimized. If so, it passes the matching oneof the first and second video signals to the output without modification(or with a limited modification that addresses ambient conditions and/orestimated adaptation of viewers as described herein). Otherwiseinterpolation/extrapolation is performed as described above, forexample.

Some embodiments provide fixed display management units which generateoutput video data based on interpolating/extrapolating from two or moresets of input video data in a manner that is independent of the imagecontent of the video data. For example, the output video data may haveattribute values that are a weighted combination of correspondingattribute values in the input video data. Weights may be determined atleast in part based upon explicit and/or assumed differences betweensource parameters (e.g. gamut, dynamic range) and target parameters(e.g. gamut, dynamic range).

In some such embodiments the display management unit may have two ormore operational modes. Input video data may be mapped to output videodata differently in the different operational modes. In some cases thedifferent operational modes may be selected automatically at least inpart based upon target display capability (e.g., capability of thetarget display indicated by display characteristics retrieved from adisplay characteristic database or received from a target display). Forexample, in some embodiments, the display management unit is configuredto receive display characteristics indicative of target displaycapability and to determine whether the target display capabilitydefines an available valid gamut and also what is the relationship ofthe target display capability to the capability required to display thehigher-quality input video data. In one example of such an embodiment:

-   -   If the target display capability is not provided or invalid, the        display management unit passes through the lower-quality legacy        video data (such as a Rec. 709 video stream). This ensures        backwards compatibility.    -   If the target display is capable of displaying the        higher-quality input video data then the display management unit        may pass through the higher-quality video data unmodified.        Alternatively, where the target display is more than capable of        displaying the higher-quality input video data, the display        management unit may extrapolate from the video data to take        advantage of the full capabilities of the target device.    -   If the target display is incapable of displaying the        higher-quality input video data then the display management unit        may interpolate between the input video data as described above,        for example.

In some embodiments, the display management unit is configured toselectively modify an incoming video signal. The selection may decidewhether or not to modify values in the video signal or, in someembodiments, how to modify values in the video signal. Selection mayresult in some values being modified while other values of the same type(i.e. other values for the same attribute) are not modified or aremodified in a different manner. Selection may be based on any or anycombination of a number of criteria. For example:

-   -   Selection may be based on the magnitude of a value. For example,        certain color or luminance values or ranges of values may be        identified as values that ought not to be modified. A display        management unit may be configured to apply one or more criteria        to each value and to modify the value only if the value        satisfies the one or more criteria. This type of selection may        be applied, for example, to avoid modification of flesh tones,        grass tones, sky tones or other tones which may be designated as        “protected colors”.    -   Selection may be based upon a location of a pixel with which        values are associated. For example, certain areas of an image        may be identified as selected areas for which values may be        modified while other areas of the image may be identified as        non-selected areas for which values ought not to be modified or        ought to be modified differently. Modifications to pixel values        may be made, or not, depending on whether the pixels lie within        the selected areas or the non-selected areas. For example, a        display management unit may receive information specifying the        locations of light sources in an image (or may determine the        location of light sources by analyzing the image). The display        management unit may selectively modify luminance values for        pixels in the immediate neighborhoods of the light sources while        not altering luminance values for pixels located away from the        light sources. The display management unit may modify luminance        values for pixels in the immediate neighborhoods of the light        sources in one manner while altering luminance values for pixels        located away from the light sources in some other manner For        example, luminance for pixels located in the immediate        neighborhoods of light sources may be made relatively higher        than luminance for pixels located away from light sources.    -   Selection may be based on an assessment of whether or not        mapping of a value would likely result in a visible artefact or        a difference that would stand out in an objectionable way when        viewed by a viewer. In some embodiments, evaluation of whether        or not to select a value for modification may be based on a        computational model of the human visual system.

In some embodiments the display management unit is configured orconfigurable to operate in an adaptive mode in which one or moremappings are determined, at least in part, on the basis of image contentin the incoming video signal(s). In an example of such an embodiment, adisplay management unit is configured to apply a mapping for oneattribute based on values of one or more other attributes. For example,a luminance mapping for a pixel may be selected or determined in partbased on color values for the pixel. For another example, a meanluminance of an image reproduced on a target display device over sometime frame may be used to estimate the adaptation of the human visualsystem of a viewer and one or more mappings may be based in part on theestimated adaptation. In another example of such an embodiment, adisplay management unit is configured to apply a mapping for oneattribute which is selected based on whether or not the pixel isidentified as corresponding to a light source (for example by a localanalysis of the image or by metadata identifying light sourcelocations). In some embodiments, a display management unit varies amapping continuously or stepwise such that the mapping can vary betweena mapping as shown in FIG. 2A to a mapping as shown in FIG. 2C, forexample.

In some embodiments, the mapping applied by a display management unitvaries with time. For example, in some embodiments a display managementunit receives inputs from sensors located at or near the display thatmonitor ambient conditions and provide ambient information to thedisplay management unit, which in turn controls mapping of values frominput to output video signals based at least in part on the ambientinformation. Ambient information may be used, for example, indetermining mappings based on estimates of adaptation of the humanvisual systems of observers, estimates of the effective range of thedisplay (e.g., black level) as affected by reflectance of ambient lightfrom the screen, and the like.

Adaptation of the human visual system to ambient conditions may beestimated by the display management unit by modeling adaptation of thehuman visual system to ambient conditions. The modeled adaptation for agiven point in time will, in general be a function of ambientinformation indicative of past ambient conditions. The modeledadaptation may take into account light emitted by a display receiving avideo signal output by the display management unit instead of or as wellas other ambient light at a viewing location. In some embodiments,ambient light may be estimated based on the video data in a video signaloutput by the display management unit. For example, adaptation of thehuman visual system may be modeled based on one or more of: the meanluminance of pixels in frames of video data over some interval and/orambient lighting as measured by one or more sensors on a display or inthe vicinity of the display. In some embodiments, adaptation of humanvisual systems is estimated based on an estimated average luminance ofthe display (which is a function of the displayed images). In otherembodiments, adaptation of video data to ambient conditions may beperformed locally, either by the display itself, or by a controller, settop box, or the like coupled to the display.

In some embodiments a luminance mapping is varied depending on anestimated level of dark-adaptation. For a high level of dark-adaptation,the mapping may be adjusted such that luminances are mapped to smallervalues whereas for lower levels of dark adaptation the mapping may beadjusted such that luminances are mapped to higher values. A level ofdark adaptation may be estimated, for example, by integrating anestimated light exposure over a characteristic adaptation period. Theestimated light exposure may be determined, for example based on one ormore of: a measure of the illumination provided by display of the videofor example; the average or mean luminance of pixels in frames of videodata; and ambient lighting as measured by one or more sensors located ator near the display.

FIG. 5 is a block diagram illustrating functional blocks of a displaymanagement unit 10B. Display management unit 10B represents pixel colorsinternally in a different format from the input and output videosignals. Display management unit 10B has an input 75 that can carry avideo signal 76 and an output 77 that can carry an output video signal78.

In the illustrated embodiment, input and output video signals 76 and 78have formats that represent colors as tristimulus values (XYZ) anddisplay management unit 10B represents colors internally in an IPT colorspace. In some embodiments, input signal 76 is in a LogYu′v′ format andoutput signal 78 is in a R′G′B′ (gamma encoded RGB) format. In the IPTcolor space, I, P and T coordinates represent lightness, red-greenchannel and yellow-blue channel values. A format converter 79A convertsinput video signal 76 to the internal color space. A format converter79B converts a processed video signal to the color space of output videosignal 78. Format converter 79B may select the color space based atleast in part on information stored in a display characteristicsdatabase 80C.

Display management unit 10B has a mapping unit 80. In the illustratedembodiment, mapping unit 80 comprises tone mapping unit 80A and gamutmapping unit 80B. Tone mapping and gamut mapping may be combined orimplemented separately, as shown. Mapping values of different attributesof the input video signal may be performed sequentially or in parallel.

In some embodiments, display management unit 10B includes an optionalmetadata extractor 79C. Metadata extractor 79C extracts metadataincluded in input signal 76 and provides the extracted metadata tomapping unit 80.

Display management unit 10B has a light source detector 81. Light sourcedetector 81 identifies light sources in images carried by the incomingvideo data and provides information about such light sources to mappingunit 80. Display management unit 10B may optionally or in thealternative receive information regarding the locations of light sourcesin metadata embedded in the incoming video signal 76, or receive suchinformation from a separate source. In some embodiments, differentmappings of one or more of tone and color are provided for pixelsassociated with light source locations than for pixels not associatedwith light source locations. In some embodiments, mappings areimplemented by values in one or more look up tables and, for at leastone mapping (e.g. for mapping luminance), separate lookup tables may beprovided for use in mapping pixels associated with light sourcelocations and for pixels not associated with light source locations.

Display management unit 10B may also optionally have a user input 82and/or a video processor input 83, which may be used to respectivelyprovide user control and/or video processor control of various aspectsof the mappings performed by mapping unit 80. In some embodiments,mapping unit 80 uses a display class and/or individual displayparameters received at video processor input 83 to retrievecorresponding information from display characteristics database 80C foruse in controlling the mappings.

The mappings performed by mapping unit 80 may be controlled by anoptional control block 84 in some embodiments. Control block 84comprises an ambient light compensation block 85 that is connected toreceive one or more inputs 85A from ambient light sensors located at ornear the display and to output information 85B indicative of ambientlight at a viewing area. Ambient light information 85B is provided to ahuman visual system (HVS) adaptation modeler 86 that provides as anoutput an estimation 86A of the sensitivity of the human visual systemto light.

Estimation 86A and ambient light information 85B are provided to a colorappearance controller 88. Color appearance controller 88 may comprise afuzzy logic controller, a neural network, a programmable controller,hard-wired logic, configurable logic circuits or the like. In someembodiments, color appearance controller 88 may model how certain colorsand/or tones are likely to be perceived by human viewers.

Color appearance controller 88 controls the operation of mapping unit 80for example by changing fixed mappings, mapping functions, parametersaffecting mappings and/or selecting values to be affected by mappings(for example, by providing values stored in look-up tables). In someembodiments, color appearance controller is configured to alter mappingsby one or more of:

-   -   selecting one of a plurality of look-up tables to perform a        mapping;    -   writing different values into a lookup table used to perform the        mapping;    -   writing new parameter values to one or more registers or memory        locations that specify selection of or behavior of a mapping        algorithm implemented in hardware or software;    -   etc.        These are examples only.

In some embodiments, color appearance controller 88 outputs one or moremasks 88A that define which pixels of an image are to be affected (ornot affected) by mappings performed by mapping unit 80 and mappingcontrol parameters 88B that control the mappings performed by mappingunit 80. In some embodiments, masks and/or control parameters mayadditionally or alternatively be specified by metadata.

In some embodiments, mappings are set according to a plurality ofdifferent inputs. for example:

-   -   an initial mapping may be performed and an output from the        initial mapping may be subjected to further mapping by a mapping        controlled by a color appearance controller; the initial mapping        may be accomplished by a fixed mapping,        interpolation/extrapolation between first and second video        signals, a mapping controlled by metadata, a mapping determined        in response to parameters specifying capabilities of a target        device or the like;    -   a mapping may be controlled in response to multiple different        inputs;    -   etc.

In some embodiments, color appearance controller 88 is configured toselect mappings that result in increased contrast in cases where ambientlight is relatively bright. In some embodiments, color appearancecontroller 88 is configured to select mappings to shift colors toward acolor point of ambient light indicated by ambient light information 85B.

FIG. 6 shows a video pipeline 100 wherein source video 102 produced by acreator of video content is provided to one or more display managementunits 104. In the illustrated embodiment, display management unit(s) 104distribute modified video signals through encoders 106 to displays 108over an electronic distribution network 107. In other embodiments,encoding functionality is incorporated into display management unit(s)104, and separate encoders are omitted. In some embodiments, displays108 have embedded decoding functionality. In other embodiments, decodingof video signals sent over electronic distribution network 107 isprovided by devices external to displays 108, such as, for example, settop boxes or the like.

Display management unit(s) 104 are configured to generate differentmodified video signals for a plurality of different types of display108. In some embodiments, a different display management unit 104 isprovided for each of a plurality of classes of displays 108. Forexample, in the FIG. 6 example, display management unit 104A providesvideo data optimized for an “Advanced” class of displays 108A throughencoder 106A, display management unit 104B provides video data optimizedfor a “Rec.709” class of displays 108B through encoder 106B, and displaymanagement unit 104C provides video data optimized for a “Mobile” classof displays 108C through encoder 106C. In other embodiments, a singledisplay management unit 104 provides video data optimized for aplurality of different classes of displays, as indicated by the dashedbox around display management units 104A-C in FIG. 6. Likewise, in someembodiments, a single encoder may encode video data for a plurality ofdifferent classes of displays. Display management unit(s) 104 may alsoprovide video data optimized for additional and/or different classesthan those shown in the FIG. 6 example.

Display management units 104 receive identifying information fromdisplays 108, and use the identifying information to retrievecorresponding characteristics from a display characteristics database103. Display characteristics database 103 may indexed by display classin some embodiments. In some embodiments, display management units 104receive a display class from each display and retrieve correspondingclass characteristics from database 103 for optimizing video data forthat class of display.

In some embodiments, a display management unit 104 may also receiveindividual parameters from a display 108 and retrieve correspondinginformation from display characteristics database 103. In someembodiments, display management unit 104 uses the information foradjusting video data optimized for a general class of displays to bettersuit a display having those individual parameters. In some embodiments,display management unit 104 uses the information for sendingsupplemental data in addition to a modified video data optimized for ageneral class of display. In some embodiments the modified video dataand the supplemental data may be combined to conform to a standard videodata format. For example, in some embodiments, video data optimized forRec. 709 displays may be combined with supplemental data for use by adisplay 108B which provides individual parameters to display managementunit 104B and encoded by encoder 106B to conform to a standard Rec. 709signal, such that the modified video data and supplemental data may beelectronically distributed together by devices configured to handlestandard Rec. 709 signals.

FIG. 7 shows a video distribution system 110 wherein source video 112which has been encoded is distributed through a transcoder 114 and overan electronic distribution network (not specifically enumerated) to adisplay 118. Transcoder 114 comprises a decoder 115 configured toreceive encoded source video data and provide decoded source video datato a display management unit 116. Display management unit 116 generatesmodified video data and provides it to an encoder 117, which in turnencodes the modified video data for distribution to display 118 over theelectronic distribution network. In some embodiments, display managementunit 116 receives identifying information from display 118 and retrievescorresponding display characteristics from a display characteristicsdatabase 113 for use in generating the modified video data.

FIG. 8 shows a server-based display management system 120 according toan example embodiment. In some embodiments, display management system120 comprises an optional version creation block 122. Block 122comprises a content version generator 124 which receives one or moreinput video signals from a video content provider and generates aplurality of versions 126 of video content. Each version 126 isconfigured to be provided to one of a plurality of different classes ofdisplay. In some embodiments, content version generator 124 may employone or more tone and/or color maps such as those described above foreach class of display to generate a version of the video content forthat class. In some embodiments, content version generator may retrievetone and color maps from a display characteristics database 128. In someembodiments, content version generator 124 receives two versions ofvideo content from the video content provider andextrapolates/interpolates as described above based on the two receivedversions to generate a version for each class of display. In someembodiments, the video content provider may provide versions 126, andblock 122 may be omitted.

Versions 126 are provided to a selection and optimization block 130.Block 130 comprises a content version selector 132 which receives adisplay class 134 from a local display control 146 and selects acorresponding selected version 136 from among versions 126 based on thereceived display class 134. In some embodiments, content versionselector 132 may retrieve corresponding display characteristics fromdisplay characteristics database 128 based on the received display class134.

A content optimizer 138 receives selected version 136 from contentversion selector 132. Content optimizer 138 may also receive individualparameters 140 for a display from local display control 146 whichindicate how that display differs from a general class of display. Whencontent optimizer 138 receives individual parameters 140, contentoptimizer 138 retrieves corresponding display characteristics fromdisplay characteristics database 128 for use in generating adjustmentdata 142. Selected version 136 and adjustment data 142 (if present) areprovided to an encoder 144 which encodes selected version 136 andadjustment data 142 for distribution over an electronic distributionnetwork to local display control 146. Adjustment data 142 may be used bylocal display control 146 to adjust selected version 136 for optimalplayback on the particular display controlled by local display control146.

FIG. 9 is a flowchart showing a method 150 of providing video contentfrom a server-based display management system to a local display controlwhich controls the operation of a display on which the video content isto be viewed. At step 152 the display management system receives displayinformation. Display information may, for example, be received directlyfrom the local display control in real time. At step 154 the displaymanagement system determines a class of the display on which the videocontent is to be viewed. At step 156 the display management systemselects a class-specific version of the video content which is optimizedfor the determined class of display. At step 158 the display managementsystem determines if the display on which the video content is to bedisplayed has any deviations from the general class of display whichwould warrant adjustment of the class-specific version. Determiningdeviations may comprise, for example, receiving individual parametersfrom the local display control in real time. If there are deviationsfrom the class (step 158 YES output), method 150 proceeds to step 160where the display management system determines individual adjustmentdata to provide to the display along with the class-specific version. Insome embodiments, determining individual adjustment data may compriseusing received display information to query a display characteristicsdatabase. After determining individual adjustment data at step 160, orif there are no deviations from the class (step 158 NO output), method150 proceeds to step 162 where the display management system providesthe class-specific version of the video content along with anyindividual adjustment data to the local display control. In someembodiments, where individual adjustment data is determined, theclass-specific version is modified based on the individual adjustmentdata at step 162 and the display management system provides the modifiedclass-specific version of the video content to the local displaycontrol. In some such embodiments, individual adjustment data is notsent to the local display control.

In some embodiments, the invention may be characterized as a displaymanagement unit configured to provide a modified video signal fordisplay on a target display over an electronic distribution network. Theunit may access information regarding the target display and at leastone input (e.g., video signal). The unit comprises a database interfaceconfigured to retrieve display characteristics corresponding to theinformation regarding the target display from a characteristicsdatabase, and a mapping unit configured to map at least one of tone andcolor values from the at least one input to corresponding mapped valuesbased at least in part on the retrieved display characteristics toproduce the modified video signal. The display management may beconfigured to access ambient information characterizing ambientconditions at the target display, and the mapping unit configured to mapthe at least one of tone and color values from the at least one inputvideo signal to corresponding mapped values based at least in part onthe ambient information. Display management units may form part of aserver-based display management system for providing video content to aplurality of displays. In addition to one or more display managementunits, the system includes a display characteristics database storingdisplay characteristics for a plurality of different types of displays.

In one embodiment, the invention comprises a display management unitconfigured to provide a modified video signal for display on a targetdisplay over an electronic distribution network, the display managementunit configured to access information regarding the target display andat least one input video signal, the display management unit comprising,a database interface configured to retrieve display characteristicscorresponding to the information regarding the target display from adisplay characteristics database; and a mapping unit configured to mapat least one of tone and color values from the at least one input videosignal to corresponding mapped values based at least in part on theretrieved display characteristics to produce the modified video signal.The display management unit may also be configured to access metadatacharacterizing at least one aspect of a creative intent affecting howthe video content embodied in the at least one input video signal oughtto be displayed, wherein the mapping unit is configured to map the atleast one of tone and color values from the input video signal tocorresponding mapped values based at least in part on the metadata. Invarious embodiments, including those noted directly above, the metadatais contained in the at least one input video signal and the displaymanagement unit is configured to extract the metadata from the videosignal.

The display management unit may be further configured to access ambientinformation characterizing ambient conditions at the target display,wherein the mapping unit is configured to map the at least one of toneand color values from the at least one input video signal tocorresponding mapped values based at least in part on the ambientinformation.

The display management unit may be further configured such that theinformation regarding the target display comprises a display classidentifier and wherein the database interface is configured to retrievedisplay class characteristics corresponding to the display classidentifier from the display characteristics database, and wherein themapping unit is configured to map the at least one of tone and colorvalues from the at least one input video signal to corresponding mappedvalues based at least in part on the retrieved display classcharacteristics.

The display management unit may be yet further configured such that theinformation regarding the target display comprises information regardingat least one individual parameter regarding the target display, whereinthe database interface is configured to retrieve adjustment informationcorresponding to the received at least one individual parameter from thedisplay characteristics database, and wherein the mapping unit isconfigured to map the at least one of tone and color values from theinput video signal to corresponding mapped values based at least in parton the retrieved adjustment information.

The display management unit may still yet further be configured suchthat at least one video input signal comprises a first input videosignal optimized for a first display and a second input video signaloptimized for a second display different from the first display, whereinthe mapping unit is configured to generate mapped values by at least oneof, interpolating the at least one of tone and color values from thefirst and second input video signals, extrapolating the at least one oftone and color values from the first and second input video signals, andpassing without modification the at least one of tone and color valuesfrom a selected one of the first and second input video signals. Themapping unit may be configured to determine from at least one of thereceived information regarding the target display and the retrieveddisplay characteristics whether a capability of the target displaymatches a capability of the displays for which the first and secondinput video signals were optimized, and, when the capability of thetarget display matches a capability of a matching one of the displaysfor which the first and second input video signals were optimized, togenerate mapped values by passing without modification the at least oneof tone and color values from the matching one of the first and secondvideo signals.

The display management unit may be even further configured such that atleast one input video signal comprises a legacy video version and asecond video version of quality higher than the legacy video version,wherein the mapping unit is configured to generate mapped values by,when the retrieved display characteristics do not indicate a validcapability of the target display, passing without modification the atleast one of tone and color values from the legacy video version, whenthe retrieved display characteristics indicate that the target displayis capable of displaying the second video version, one of passingwithout modification the at least one of tone and color values from thesecond video version and extrapolating the at least one of tone andcolor values from the legacy video version and second video version, andwhen the retrieved display characteristics indicate that the targetdisplay is incapable of displaying the second video version,interpolating the at least one of tone and color values from the legacyvideo version and the second video version.

In other embodiments, which may include any number of the abovefeatures, the invention comprises a display management unit configuredto provide a modified video signal for display on a target display overan electronic distribution network, the display management unitconfigured to access information regarding the target display and atleast one input video signal, the display management unit comprising, adatabase interface configured to retrieve display characteristicscorresponding to the information regarding the target display from adisplay characteristics database, and a mapping unit configured to mapat least one of tone and color values from the at least one input videosignal to corresponding mapped values based at least in part on theretrieved display characteristics to produce the modified video signal.The mapping unit may be configured to selectively map the at least oneof tone and color values from the at least one input video signal tocorresponding mapped values based at least in part on magnitudes of theat least one of tone and color values. Alternatively, for example, themapping unit may be configured to selectively map the at least one oftone and color values from the at least one input video signal tocorresponding mapped values based at least in part on locations ofpixels corresponding to the at least one of tone and color values inimage frames of the input video signal. In yet another examplealternative the mapping unit may be configured to selectively map the atleast one of tone and color values from the at least one input videosignal to corresponding mapped values based at least in part on anassessment of whether mapping the values would likely result in anartefact visible to a human observer. In yet still another alternative,the mapping unit may be configured to map the at least one of tone andcolor values from the at least one input video signal to correspondingmapped values based at least in part on image content in the input videosignal, and the mapping unit may also configured to any of: map tonevalues from the at least one input video signal to corresponding mappedvalues based on at least one corresponding color value, map the at leastone of tone and color values from the at least one input video signal tocorresponding mapped values based at least in part on a mean luminanceof pixels in the output video signal over a period of time, map the atleast one of tone and color values from the at least one input videosignal to corresponding mapped values based at least in part on anestimated mean illumination provided by the target display in thedisplay of the output video signal over a period of time, the estimatedmean illumination based at least in part on luminance of pixels in theoutput video signal over a period of time and retrieved displaycharacteristics, and map the at least one of tone and color values fromthe at least one input video signal to corresponding mapped valuesaccording to a mapping that varies over time.

Mapping that varies over time may be based, for example, at least inpart on an estimated adaptation of the human visual system. Estimatedadaptations of the human visual system may be derived from any of, forexample, a model of adaptation of the human visual system, the modelcomprising a function of ambient information, ambient information based,at least in part, on a mean luminance of pixels in the output videosignal over a period of time, and ambient information based, at least inpart, on an estimated mean illumination provided by the target displayin the display of the output video signal over a period of time, theestimated mean illumination based at least in part on luminance ofpixels in the output video signal over a period of time and retrieveddisplay characteristics.

In one embodiment, the present invention comprises a server-baseddisplay management system for providing video content to a plurality ofdisplays, the system comprising, a display characteristics databasestoring display characteristics for a plurality of different types ofdisplays, and, one or more display management units, each displaymanagement unit configured to provide a modified video signal fordisplay on a target display over an electronic distribution network,each display management unit configured to access information regardingthe target display and at least one input video signal and comprising, adatabase interface configured to retrieve display characteristicscorresponding to the information regarding the target display from thedisplay characteristics database, and, a mapping unit configured to maptone and/or color values from the at least one input video signal tocorresponding mapped values based on the retrieved displaycharacteristics to produce the modified video signal for the targetdisplay. The mapping unit may include any of the features describedelsewhere herein to provide an appropriate mapping based on any aspectof the invention.

In one embodiment, the invention comprises a transcoder configured toreceive an encoded input video signal and provide an encoded modifiedvideo signal, the transcoder comprising, a decoder configured to decodethe encoded input video signal and produce a decoded input video signal,a display management unit configured to receive the decoded input videosignal and configured to map tone and/or color values from the inputvideo signal to corresponding mapped values to produce a modified videosignal, and, an encoder configured to receive the modified video signaland encode the modified video signal.

In one embodiment, the present invention comprises a method in displaymanagement unit for producing a modified video signal for display on atarget display, the method comprising, accessing an input video signal,accessing information regarding the target display, retrieving displaycharacteristics corresponding to the information regarding the targetdisplay from a display characteristics database, mapping at least one oftone and color values from the input video signal to correspondingmapped values based at least in part on the retrieved displaycharacteristics to produce the modified video signal, and, providing themodified video signal to the target display over an electronicdistribution network.

In one embodiment, the present invention comprises a display managementunit configured to provide a modified video signal for display on atarget display over an electronic distribution network, the displaymanagement unit configured to receive at least one input video signaland metadata characterizing at least one aspect of a creative intentaffecting how the video content embodied in the at least one input videosignal ought to be displayed, and comprising, a mapping unit configuredto map at least one of tone and color values from the at least one inputvideo signal to corresponding mapped values based at least in part onthe metadata, and, a video output connectable to provide the modifiedvideo signal to the target display over an electronic distributionnetwork. The metadata may be contained, for example, in the at least oneinput video signal.

In one embodiment, the present invention comprises a display managementunit configured to provide a modified video signal for display on atarget display over an electronic distribution network, the displaymanagement unit configured to access at least one input video signal andinformation regarding the target display, the information regarding thetarget display comprising a display class identifier identifying adisplay class of the target display, the display management unitcomprising, a content version generator configured to generate aplurality of versions of video content from the at least one input videosignal, each version configured to be provided to one of a plurality ofdifferent classes of display, and a content version selector configuredto select a selected version of the plurality of versions of videocontent based on the display class identifier, wherein the displaymanagement unit is configured to generate the modified video signalbased on the selected version. The display management unit may furthercomprise a database interface configured to retrieve display classcharacteristics corresponding to each of the plurality of differentclasses of display from a display characteristics database, wherein thecontent generator comprises a mapping unit configured to map at leastone of tone and color values from the at least one input video signal tocorresponding version-specific mapped values based at least in part onthe retrieved display class characteristics. The at least one videoinput signal may comprise, for example, a first input video signaloptimized for a first display class and a second input video signaloptimized for a second display class different from the first displayclass, wherein the mapping unit is configured to generateversion-specific mapped values by at least one of, interpolating the atleast one of tone and color values from the first and second input videosignals, extrapolating the at least one of tone and color values fromthe first and second input video signals, and passing withoutmodification the at least one of tone and color values from a selectedone of the first and second input video signals. The informationregarding the target display may comprise information regarding at leastone individual parameter regarding the target display, wherein thedatabase interface is configured to retrieve display characteristicscorresponding to the at least one individual parameter, and comprise acontent optimizer configured to generate individual adjustment databased on the retrieved display characteristics and to map at least oneof tone and color values from the selected version to correspondingmapped values based at least in part on the individual adjustment data.The information regarding the target display may comprise informationregarding at least one individual parameter regarding the targetdisplay, wherein the database interface is configured to retrievedisplay characteristics corresponding to the at least one individualparameter, and comprise a content optimizer configured to generateindividual adjustment data based on the retrieved displaycharacteristics, and the display management unit configured to combinethe selected version and the at least one of the at least one individualparameter to conform to a standard video format.

While a number of exemplary aspects and embodiments have been discussedabove, those of skill in the art will recognize certain modifications,permutations, additions and sub-combinations thereof. This disclosure isintended to be interpreted to encompass all such modifications,permutations, additions and sub-combinations as are within its truespirit and scope.

What is claimed:
 1. A display management unit for a target displaycomprising: a decoder, the decoder receiving: (1) source video data andassociated metadata, wherein the source video data is graded by acontent provider according to dynamic range and color gamut to comprisea first range of video parameters, and (2) data regarding the targetdisplay, comprising the target display characteristics and the targetdisplay ambient conditions, the target display characteristics relatingto a second range of video parameters; a video mapping unit, the videomapping unit mapping the source video data and associated metadata ofthe first range of video parameters to be displayed upon the targetdisplay to preserve the creative intent of the content provider; anencoder, the encoder receives the mapped source video data and outputs amodified video signal for rendering upon the target display.
 2. Thedisplay management unit of claim 1, wherein further the encoder outputsenhanced video information, the enhanced video information comprisingone of high spatial frequency information, enhanced dynamic rangeinformation.
 3. The display management unit of claim 2, wherein furtherthe encoder outputs metadata relating to the modified video content. 4.The display management unit of claim 1, wherein further the modifiedvideo signal is a function of the maximum and minimum brightness, colorgamut, and other characteristics of the target display.
 5. The displaymanagement unit of claim 4, wherein further, if the first range of videoparameters of the source video data is greater than the second range ofvideo parameters of the target display, the video mapping unit maps thesource video data to determine the parameters at which the source videodata starts to be compressed and at which the maximum output value is inthe modified video signal.
 6. The display management unit of claim 4,wherein further, if the first range of video parameters of the sourcevideo data is less than the second range of video parameters of thetarget display, the video mapping unit maps the source video data todetermine the parameter at which the source video data starts to beexpanded.
 7. The display management unit of claim 1, wherein the decoderfurther comprises an extractor, the extractor extracts attributes of thesource video data for translation of the attributes for the modifiedvideo data.
 8. The display management unit of claim 7, wherein one ofthe attributes extracted from the source video data comprises colorspace data of the source video data.
 9. The display management unit ofclaim 1, wherein the metadata that accompanies the source video datacarries information about the content grading system used to grade thecontent.
 10. The display management unit of claim 1, wherein further thedisplay management unit employs the metadata to transforms source videodata to produce an optimized output for display on the target device.11. A method for image processing on a display management unit for atarget display, comprising: receiving source video data and associatedmetadata, wherein the source video data is graded by a content provideraccording to dynamic range and color gamut to comprise a first range ofvideo parameters; receiving data regarding the target display,comprising the target display characteristics and the target displayambient conditions, the target display characteristics relating to asecond range of video parameters; mapping the source video data andassociated metadata of the first range of video parameters to bedisplayed upon the target display to preserve the creative intent of thecontent provider; encoding a modified video signal for rendering uponthe target display, modified video signal a function of the mappedsource video data and associated metadata.
 12. The method of claim 11,wherein encoding a modified video signal further comprises encodingmodified video signal as a function of the maximum and minimumbrightness, color gamut, and other characteristics of the targetdisplay.
 13. The method of claim 11, wherein further, if the first rangeof video parameters of the source video data is greater than the secondrange of video parameters of the target display, mapping the sourcevideo data further comprises determining the parameters at which thesource video data starts to be compressed and at which the maximumoutput value is in the modified video signal.
 14. The method of claim11, wherein further, if the first range of video parameters of thesource video data is less than the second range of video parameters ofthe target display, mapping the source video data further comprisesdetermining the parameter at which the source video data starts to beexpanded.
 15. The method of claim 11, further comprising extractingattributes of the source video data for translation of the attributesfor the modified video data.
 16. The method of claim 15, wherein furtherone of the attributes extracted from the source video data comprisescolor space data of the source video data.
 17. The method of claim 11,wherein further the metadata that accompanies the source video datacarries information about the content grading system used to grade thecontent.
 18. The method of claim 11, further comprising employing themetadata to transforms source video data to produce an optimized outputfor display on the target device.